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ABSTRACT 

A longitudinal study was planned to track a class in 
a prekindergarten program to determine program effectiveness. There 
are many problems in conducting a longitudinal study, most of which 
revolve around the long-term commitment required. A carefully 
designed information system is important. For . the study in question, 
computer software and hardware were not available. Attrition of 
subjects is a central problem in longitudinal studies, but changes in 
data-gather ing personnel can grossly alter the information management 
system. Data collection activities need to be institutionalized from 
the beginning of the study, with clear definitions and plans from the 
outset. The purpose and importance of the longitudinal study must be 
understood from the beginning, and teachers at all levels should be 
made to feel part of the study. (Contains 20 references.) (SLD) 
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The primary purpose of the longitudinal study was to 
track the current class of a pre-kindergarten program to 
determine program effectiveness. Evaluations of early 
childhood education generally look at the relationship 
between stated goals of the program and the actual 
outcomes of the first year. Of course, evaluations 
looking for evidence of program effectiveness can deliver 
usable data in a current year through testing, parent 
interviews or classroom observations. However, some data 
can only be collected through a longitudinal design. 

Such trends as the loss of immediate effects or the 
awakening of latent effects are examples of what can be 
recognized only through longitudinal studies. If the 
right longitudinal data is collected, trends can be 
spotted, programs continuously upgraded and modified and 
the school stakeholders better served. 

Perspective 

According to Darlington and Lazar (1982) , 
longitudinal studies can provide a holistic, panoramic 
understanding of developmental data. The early 
longitudinal studies of Headstart discovered a ''sleeper” 
effect among the Headstart graduates demonstrating 
greater than expected achievement in later years after 
early effects had declined. Such a discovery justified 
the continued support of Headstart, yet could not have 
been found except through longitudinal data. 



The problems of a longitudinal study are many. The 
attrition of subjects is an easily recognizable one. 
Subjects move away, leave school or are otherwise 
unreachable. Changes in data gathering personnel is 
another problem. Original personnel retire or go on to 
other jobs or lose interest. For the evaluation of new 
programs efficient data collection, including recognition 
of important data, needs to be established from the 
beginning. The longitudinal study, by nature, needs a 
long term commitment. This commitment must be 
institutionalized in order to pass from current personnel 
to future successors. 

Description of Study 

The longitudinal study designed by the authors was 
interested in assuring that proper data was collected in 
an efficient manner. To accomplish this, interviews were 
conducted with district and school personnel. Also, 
current records and record keeping processes were 
reviewed. Personnel's input was needed to determine the 
most logical school office to have the responsibility of 
data collection. It was important to understand what is 
currently collected and how it is stored. 

The longitudinal follow-up study, by nature, is 
designed as an ongoing process. The initial data of 
project will hardly produce any significant results 
within the first few years. On the other hand, 
evaluators, administrators, and program teachers may 



occasionally want to look at the collected data in order 
to gain insights into the strength and weakness of the 
program. In this regard, a carefully designed 
information management system would prove most 
instrumental not only because of its ability to store, 
retrieve and update data but because of its various 
functions to facilitate data analysis. 

There are a number of data sources available to 
develop such an information management system. An effort 
was made to take advantage of as many of these sources as 
possible. Parent input was sought through designed 
interviews. Teacher questionnaires were especially 
designed to verify information from other sources. 
However, the teacher reports could also produce 
qualitative information that may be lost in student 
records. Student self-reports would come from exit 
interviews with those dropping out. Guidance office 
questionnaires could provide information about 1) records 
of student participation in extracurricular and community 
activities; 2) records of disciplinary action; 3) guidance 
office reports; 4) administrative reports. Results of 
various state and standardized tests at appropriate grade 
levels would an intricate part of the data management 
system. 

A computerized tracking system which could follow 
the progress of individual students through all grade 
levels would be ideal. Data could be gathered more 
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efficiently about each student' demographic information, 
socioeconomic characteristics, school performance, social 
behavior, and psychological states (Webster & Larson, 
1991) . Interactions among the variables collected could 
be readily analyzed. The idea was for a basic program 
producing simple graphs as analysis of data. 

In an effort not to reinvent the wheel, a review of 
available data processing software was conducted. We 
were particularly interested in software specifically 
developed for aiding longitudinal studies. No such 
software was available at the time. As the school was 
not ready for computerization, the program was not 
developed. 

Point of View 

Follow-up studies are often conducted to make an 
internal or external evaluation of an educational 
program, or some aspect of it (Gay, 1987) . When these 
studies are carried on longitudinally, a particular 
individual or group of individuals will be followed over 
a substantial period of time to discern the effects of 
the evaluated (Scriven, 1991) . In this way, the merit, 
value or worth of the program can be determined, and the 
subsequent policy analysis, program management, and 
political action facilitated (Borg & Gall, 1989). 

Over the last few decades or so preschool prevention 
programs have provided services to large numbers of 
disadvantaged American children so as to reduce the 
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number of students who are retained at higher grade 
levels, lower the dropout rate, and improve the chances 
for at risk students with regard to school success. When 
investigators began follow-up studies of such programs 
two or three years after their termination, the program 
children showed little sustained advantage resulting from 
the intervention (Hetherington & Parke, 1986) . When 
evaluators did longitudinal follow-up studies instead, 
however, evidence of the positive effects of preschool 
intervention programs immediately started to accumulate 
(Lazar & Darlington, 1982; Hans, 1987; Sevigny, 1987; Lee 
et al. , 1989) . 

Importance to Field 

The pre-kindergarten program of this study, first 
established in 1989, was designed to meet the needs of 
three- and four- year-olds who have been identified as 
at-risk for being unprepared for school. Based on a 
curriculum of goals, objectives and activities organized 
around the children's full development, this program has 
strived to create a warm, loving, stimulating 
environment. Its goals have been 1) to support each 
child's own learning priorities, 2) to allow children to 
acquire a solid sense of security, positive self-esteem 
and 3) to develop in the child a long term love of 
learning as they progress through school. 

In order to find out the true effectiveness of an 
early intervention program with such long-term goals, a 



7 



longitudinal follow-up study should be organized to 
answer the question: Does a child's growth continue to be 
enhanced after the intervention period (Hans, 1987)? It 
is usually desirable in a follow-up study to attain data 
on a comparison group equated at least crudely to the 
"experimental" cases on the obvious demographic variables 
(Cronbach, 1983) . Therefore, the proposed follow-up 
study should answer another question as well: How do the 
program children compare with control children in terms 
of their social, cognitive, and emotional development? 

A particular concern of the district was reflected 
in the longitudinal study and goes beyond the judgment of 
merit and worth of the current program. That concern is 
the previously established ineffectiveness of another 
program for disadvantaged children several years past. 

The program was a pre-first and was perceived to be 
effective for a long time. In reality a longitudinal 
study using archival data demonstrated that it was not 
fulfilling its stated purpose of preparing children for 
school. It was hoped that through the designed study, 
the right information could be effectively analyzed. The 
effect would be that the pre-kindergarten program could 
be recognized for its long range effects early enough to 
make a difference. 

Variables in Longitudinal Studies 

As noted before, longitudinal follow-up studies of 
early intervention programs generally investigate issues 



7 



O 

ERLC 



involving program participants' later school success or 
failure. Interviews with the district personnel and 
experts in the field in this reported study also helped 
to clarify additional issues of importance. These issues 
can be addressed in relation to various factors such as 
1) socioeconomic characteristics (e.g., family mobility, 
number of siblings, and number of parents residing with 
the student), 2) school performance (e.g., attendance, 
work habits, GPA, standard test scores, grade placement, 
retention, and referral to special education or social 
service), 3) social behavior (e.g., socialization 
patterns, violation of school regulations, involvement in 
the juvenile justice system) , 4) developmental status 
(e.g. , cognitive, social, and motor development) , and 5) 
psychological states (e.g., self-esteem, educational 
aspirations, and sense of efficacy) . In this reported 
study, these factors were to be updated through the 
teacher and counselor surveys and consolidated on the 
data collection form. These factors are important 
indicators of program effectiveness on the assumption 
that the effects of an early intervention program should 
be recognizable after the intervention period. 

Indicators of socioeconomic status are important 
factors to be studied since relevant literature 
demonstrates they have a close correlation with the 
conditions of children-at-risk (Lally, 1987; Rachal & 
Garbo, 1988) . In the teachers survey and the guidance 
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counselor survey, standard indicators of socioeconomic 
status such as eligibility for free lunch, parents' 
occupation, parents' education level are asked for each 
survey year. Though there is little in the literature to 
indicate the effects of changed socioeconomic status on 
the child-at-risk, this information can easily be 
obtained by a longitudinal study and would contribute to 
an enhanced understanding. 

The continual consideration of the student's 
situation was reflected, by design, in survey items 
asking the current status of other correlates of the "at- 
risk" classification. These include the current family 
structure, the number of siblings, the family mobility 
and access to social services (Webster & Lawson, 1991) . 
Because the literature indicates the child who has been 
retained is at increased risk for dropping out, this has 
been included as a variable to be studied (Shepard & 
Smith, 1989) . 

School performance is another area usually 
investigated in longitudinal follow-up studies (Sevigny, 
1987) . The factors from this area are reflected in the 
items on the teachers and the counselor surveys 
associated with academic development, school 
interventions and quartile range on standardized tests. 
These factors are important to be studied because they 
most accurately reflect the child's adaptation to the 
educational environment and values (Rachal & Garbo, 1988; 



Sevigny, 1987) . This adaptation is most notable 
objective of early intervention (Hubbell, 1983; Lazar & 
Darlington, 1982) . 

Another objective of early intervention and a 
frequent variable in longitudinal follow up studies is 
the enhancement of social behavior (Hubbell, 1983) . In 
this reported study, the measurement of this enhancement 
was to be accomplished through the items on the teachers 
and the counselor surveys in the areas of social 
development and student's behavior. In the guidance 
counselor survey, there are additional sections asking 
the student's involvement with crime or other delinquent 
behaviors. The conditions that lead the child to be 
classified as "at-risk" may also contribute to 
delinquency (Lally, 1987; Lloyd, 1978). Still, in 
analysis of the data it should be noted that the causes 
of delinquency are many and may have nothing to do with 
the presence or effectiveness of intervention (Patterson 
& Stouthamer-Loeber , 1984) . 

Developmental status across domains in early 
childhood is measured by the developmental screening 
instrument, ESI, in pre-kindergarten and/or kindergarten. 
The literature indicates these measures should have 
little correlation with measures taken after the middle 
of the kindergarten year (Bloom, 1964) . However, they 
are important to collect in a longitudinal study in order 
to plot the growth pattern of the child as a whole 



(Bloom, 1964) . Additionally, in analysis of the data, 
previously unrecognized trends or correlations may become 
apparent (Sattler, 1982) . 

The self efficacy, self esteem and other 
psychological states of the student are important to 
recognize. Some children thought to be at-risk have' 
proven to be remarkably resilient in difficult life 
circumstances (Werner, 1991) . If program participants 
demonstrate such a resiliency based on positive mental 
attitudes, such a finding would be indicative of the 
program's effectiveness (Rachal & Garbo, 1988) . Elements 
of productive psychological states were intended to be 
derived from the open questions at the end of the 
teachers and the counselor surveys. 

Institutionalized Commitment 

As noted before certain problems exist for a 
longitudinal study such as the attrition of subjects. 
However, changes in data gathering personnel can grossly 
alter the information management system. The use of 
archival data planned for the evaluation of a pre- 
existing program is fine if one can guarantee its 
efficient management after the original personnel are 
gone. There is no guarantee that the salient data has 
been gathered or accurately stored. For new programs 
efficient data collection, including recognition of 
important data, needs to be established from the 
beginning. The longitudinal study, by its nature, needs 
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a long term commitment. Again, this commitment must be 
institutionalized in order to pass from current personnel 
to future successors. This commitment has to be 
expressed in terms of certain issues. 

1) The design of the program to be evaluated needs 
a full and comprehensive description in order to better 
understand any correlational findings from the 
longitudinal study. 

2) The data collection system has to be easily 
understood by district personnel in order to be readily 
entered and accessed. 

3) There needs to be a clear plan from the 
beginning for analysis of data annually or at pertinent 
intervals. 

4) Responsibilities for data collection, analysis 
and supervision need to be fitted into already existing 
job descriptions. As personnel change, responsibilities 
can remain. 

5) The purpose and importance of the longitudinal 
study needs to be understood district wide. Teachers at 
all levels should be made to feel part of the study and 
important contributors . 

It is assumed in longitudinal studies that the task 
of data collection Vv^- 11 change hands over the years. 
However, if record keeping is already part of an 
employee's job description then data gathering task can 
be incorporated into that job description. However, the 
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dedication and seriousness shown the task by district 
professionals may make one of them the more logical 
choice. Also the accessibility of some new data may make 
it a task for a professional, e.g. the school 
psychologists . 
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